Comparative Analysis of Existing and a Novel Approach to Topic Detection on Conversational Dialogue Data

Topic detection in dialogue datasets has become a significant challenge for unsupervised and unlabeled data to develop cohesive engaging system. In this paper, we proposed semi-supervised techniques topic the conversational dataset compared them with existing techniques. The paper proposes novel approach detection, which takes preprocessed as an input performs similarity analysis TF-IDF scores bag of words technique (BOW) identify higher frequency from utterances. It then refines by integrating clustering elbow methods using Parallel Latent Dirichlet Allocation (PLDA) model detect topics. comprised comparative on Switchboard, Personachat MultiWOZ dataset. experimental results show that significantly better We also performed quantification check how accurate extracted topics are compare manually annotated data. For example, Switchboard 92.72%, Peronachat 87.31% 93.15%